Machine Learning Techniques For Classification Research Articles

Systematic review and best practices for drone remote sensing of invasive plants

Abstract Drones have emerged as a cost‐effective solution to detect and map plant invasions, offering researchers and land managers flexibility in flight design, sensors and data collection schedules. A systematic review of trends in drone‐based image collection, data processing and analytical approaches is needed to advance the science of invasive species monitoring and management and improve scalability and replicability. We systematically reviewed studies using drones for plant invasion research to identify knowledge gaps, best practices and a path toward advancing the science of invasive plant monitoring and management. We devised a database of 33 standardized reporting parameters, coded each study to those parameters, calculated descriptive statistics and synthesized how these technologies are being implemented and used. Trends show a general increase in studies since 2009 with a bias toward temperate regions in North America and Europe. Most studies have focused on testing the validity of a machine learning or deep learning image classification technique with fewer studies focused on monitoring or modelling spread. Very few studies used drones for assessing ecosystem dynamics and impacts such as determining environmental drivers or tracking re‐emergence after disturbance. Overall, we noted a lack of standardized reporting on field survey design, flight design, drone systems, image processing and analyses, which hinders replicability and scalability of approaches. Based on these findings, we develop a standard framework for drone applications in invasive species monitoring to foster cross‐study comparability and reproducibility. We suggest several areas for advancing the use of drones in invasive plant studies including (1) utilizing standardized reporting frameworks to facilitate scientific research practices, (2) integrating drone data with satellite imagery to scale up relationships over larger areas, (3) using drones as an alternative to in‐person ground surveys and (4) leveraging drones to assess community trait shifts tied to plant fitness and reproduction.

Global and local interpretability techniques of supervised machine learning black box models for numerical medical data

The most effective machine learning classification techniques, such as artificial neural networks, are not easily interpretable, which limits their usefulness in critical areas, such as medicine, where errors can have severe consequences. Researchers have been working to balance the trade-off between the model performance and interpretability. In this study, seven interpretability techniques (global surrogate, accumulated local effects, local interpretable model-agnostic explanations (LIME), Shapley additive explanations (SHAP), model agnostic post hoc local explanations (MAPLE), local rule-based explanation (LORE), and Contextual Importance and Utility (CIU)) were evaluated to interpret five medical classifiers (multilayer perceptron, support vector machines, random forests, extreme gradient boosting, and naïve bayes) using six model performance metrics and three interpretability technique metrics across six medical numerical datasets. The results confirmed the effectiveness of integrating global and local interpretability techniques, and highlighted the superior performance of global SHAP explainer and local CIU explanations. The quantitative evaluations of explanations emphasised the importance of assessing these interpretability techniques before employing them to interpret black box models.

SecureNet: Network Intrusion Detection using Machine Learning and Deep Learning Techniques

Abstract: In the ever-evolving landscape of cybersecurity, the need for robust intrusion detection systems has become paramount. This paper introduces a cutting-edge intrusion detection algorithm designed to enhance network security through the integration of advanced machine learning and deep learning methodologies. The proposed algorithm capitalizes on the strengths of both paradigms to achieve a comprehensive and adaptive approach to identifying malicious activities within a network. This research focuses on enhancing network security through the development and evaluation of a novel intrusion detection system leveraging both deep learning and traditional machine learning approaches. Utilizing the NSL-KDD dataset, we employ the Long Short-Term Memory (LSTM) model, a superior version of Recurrent Neural Networks (RNNs), and the K-Nearest Neighbors (KNN) algorithm for binary and multi-class classification of network intrusion anomalies. The LSTM model excels in capturing temporal dependencies, enabling the detection of nuanced sequential patterns, while the KNN algorithm contributes to a comprehensive classification framework. Experimental results demonstrate the effectiveness of the hybrid methodology, showcasing improved accuracy, precision, and recall compared to traditional methods. This research underscores the potential of integrating deep learning and classical machine learning techniques to bolster the capabilities of intrusion detection systems in safeguarding against evolving cyber threats

Image classification using deep and classical machine learning models on small datasets: a complete comparative

One of the most important challenges in the Machine and Deep Learning areas today is to build good models using small datasets, because sometimes it is not possible to have large ones. Several techniques have been proposed in the literature to address this challenge. This paper aims at studying the different available Deep Learning techniques and performing a thorough experimentation to analyze which technique or combination thereof improves the performance and effectiveness of the models. A complete comparison with classical Machine Learning techniques was carried out, to contrast the results obtained using both techniques when working with small datasets. Thirteen algorithms were implemented and trained using three different small datasets (MNIST, Fashion MNIST, and CIFAR-10). Each experiment was evaluated using a well-established set of metrics (Accuracy, Precision, Recall, F1, and the Matthews correlation coefficient). The experimentation allowed concluding that it is possible to find a technique or combination of them to mitigate a lack of data, but this depends on the nature of the dataset, theamount of data, and the metrics used to evaluate them.

URL Detection based on YOLO Network in Various Conditions

Object detection is a key technique in computer vision, as it is considered a necessary step in any recognition process. It is the procedure for determining the instance of the class to which the object belongs and estimating its location by displaying its bounding box. It was widely accepted that advances in object detection have generally gone through two periods: ”the traditional object detection period”, where detection was performed through classical machine learning techniques, and ”the deep learning based detection period”, where classical machine learning techniques have been completely replaced by methods based on deep neural networks. In this paper, we will focus on object detection based on deep learning. The main objective is to carry out a comparative study of three models of the YOLO family, already proven to be effective for object detection that are YOLOv3, YOLOv4, and YOLOv5 in the context of the detection of URLs in photos taken by a mobile phone. The experimental results, expressed in terms of average precision, showed the generalization ability of the three models, YOLOv3, YOLOv4, and YOLOv5. In addition, the stability of the YOLOV4 model against several difficulties added to the images.

An effective ensemble classification framework for intrusion detection system using LSTM autoencoder

In intrusion detection, the curse of dimensionality and the trade-off between maintaining a low false alarm rate and achieving a high detection rate are significant challenges. This research suggests a unique strategy based on dimensionality reduction methods to improve the performance of network intrusion detection systems (NIDS). Compressing high-dimensional network traffic data using a Long Short-Term Memory Autoencoder (LSTMAE) allows the reduced characteristics to be submitted to a classifier to identify anomalies that may indicate an attack. Using standard datasets, including Network Security Laboratory - Knowledge Discovery in Datasets (NSL-KDD), UNSW-NB15, and Canadian Institute for Cyber Security - Intrusion Detection Systems (CICIDS2017), the proposed model is tested with classifiers like Random Forest (RF) and LightGBM (Light Gradient Boosting Machines). It is hoped that by adopting this method, NIDS response times may be improved while costs associated with storing and processing data are minimized. Precision, recall, F-score, accuracy, detection rate (DR), and false alarm rate (FAR) are only a few of the performance measures used to assess the quality of the suggested models. The experimental findings show that the proposed LSTMAE model reduces prediction errors more effectively than classic machine learning techniques such as Random Forest (RF), Gradient Boosting (GB), Support Vector Machines (SVM), Deep Belief Networks (DBN), Deep Neural Networks (DNN), Autoencoder (AE), and Long Short-Term Memory (LSTM). The results also show that the proposed solution outperforms the state-of-the-art methods of detection accuracy and computing complexity using accuracy, precision, recall, F1_Score, detection rate, and FAR.

Performance Comparison of Apache Spark and Hadoop for Machine Learning based iterative GBTR on HIGGS and Covid-19 Datasets

In the realm of distributed computing frameworks, such as Apache Spark and MapReduce Hadoop, the efficacy of these frameworks varies across diverse applications and algorithms contingent upon distinctive evaluation metrics and critical parameters. This research paper diligently scrutinizes the extant body of research that compares these two frameworks concerning said evaluation metrics and parameters. Subsequently, it conducts empirical investigations to authenticate the performance of these frameworks in the context of an iterative Gradient Boosting Tree Regression (GBTR) algorithm. Remarkably, the comparative analyses in previous studies encompass a spectrum of iterative machine learning regression and classification techniques, batch processing, SQL, and Graph processing algorithms. Furthermore, numerous investigations have explored the application of machine learning algorithms encompassing logistic regression, Page Rank, K-Means, KNN, and the HiBench suite. This paper presents the comparison between the two distributed computing platforms on iterative GBTR for classification task on the HIGGS dataset from the physics domain and for the regression task on the Covid-19 dataset from the healthcare domain. The empirical findings corroborate that Apache Spark exhibits superior execution speed in iterative tasks when the available physical memory significantly exceeds the dataset size. Conversely, Hadoop outperforms Spark when dealing with substantial datasets or constrained physical memory resources.

A Comparison Study of Deep Learning Methodologies for Music Emotion Recognition.

Classical machine learning techniques have dominated Music Emotion Recognition. However, improvements have slowed down due to the complex and time-consuming task of handcrafting new emotionally relevant audio features. Deep learning methods have recently gained popularity in the field because of their ability to automatically learn relevant features from spectral representations of songs, eliminating such necessity. Nonetheless, there are limitations, such as the need for large amounts of quality labeled data, a common problem in MER research. To understand the effectiveness of these techniques, a comparison study using various classical machine learning and deep learning methods was conducted. The results showed that using an ensemble of a Dense Neural Network and a Convolutional Neural Network architecture resulted in a state-of-the-art 80.20% F1 score, an improvement of around 5% considering the best baseline results, concluding that future research should take advantage of both paradigms, that is, combining handcrafted features with feature learning.

Toward an improved ensemble of multi-source daily precipitation via joint machine learning classification and regression

Accurate estimation of precipitation at local to global scales can considerably enhance our understanding of climate system dynamics. While numerous precipitation products are available as indispensable tools for investigating precipitation and its associated processes, none can consistently provide the lowest estimation error across environmental conditions. The multiple source precipitation ensemble (MSPE) methods have been considered a vital solution. A new MSPE framework is proposed here, which simultaneously uses machine learning (ML) classification and regression techniques within an automatic workflow (MSPEaml). Six precipitation products and their ensembles based on different MSPE strategies were evaluated at 2365 gauged and 800 randomly selected ungauged sites over China. Results revealed significant precision inconsistencies among the products primarily due to their different data sources and retrieval algorithms; while MSPEaml can effectively reduce the random and classification errors of estimated precipitation according to the Kling-Gupta efficiency and Heidke skill score. The improvements demonstrated the unique features of MSPEaml, particularly the necessity of the joint use of ML classifiers and regressors and assigning spatiotemporal dynamic weights for merging precipitation data. Moreover, MSPEaml can substantially improve its generalizability through a simple binning procedure, making it applicable under more complex conditions. The varying contributions of predictor variables (indicated by Shapely values) in different ML models identified the complexity of the MSPE issue and further the importance of designing proper ML models according to specific targets. The proposed MSPE framework is expected to be a suitable solution for assembling multiple precipitation data sources with different time periods and scales.

Detecting cyberbullying using deep learning techniques: a pre-trained glove and focal loss technique.

This study investigates the effectiveness of various deep learning and classical machine learning techniques in identifying instances of cyberbullying. The study compares the performance of five classical machine learning algorithms and three deep learning models. The data undergoes pre-processing, including text cleaning, tokenization, stemming, and stop word removal. The experiment uses accuracy, precision, recall, and F1 score metrics to evaluate the performance of the algorithms on the dataset. The results show that the proposed technique achieves high accuracy, precision, and F1 score values, with the Focal Loss algorithm achieving the highest accuracy of 99% and the highest precision of 86.72%. However, the recall values were relatively low for most algorithms, indicating that they struggled to identify all relevant data. Additionally, the study proposes a technique using a convolutional neural network with a bidirectional long short-term memory layer, trained on a pre-processed dataset of tweets using GloVe word embeddings and the focal loss function. The model achieved high accuracy, precision, and F1 score values, with the GRU algorithm achieving the highest accuracy of 97.0% and the NB algorithm achieving the highest precision of 96.6%.

Analysis of selected deep features with CNN-SVM-based for bread wheat seed classification

AbstractThe main ingredient of flour is processed wheat. Wheat is an agricultural product that is harvested once a year. It may be necessary to choose the variety of wheat for growing wheat and efficient harvesting. The variety of wheat is important for its economic value, taste, and crop yield. Although there are many varieties of wheat, they are very similar in colour, size, and shape, and it requires expertise to distinguish them by eye. This is very time consuming and can lead to human error. Using computer vision and artificial intelligence, such problems can be solved more quickly and objectively. In this study, an attempt was made to classify five bread wheat varieties belonging to different cultivars using Convolutional Neural Network (CNN) models. Three approaches have been proposed for classification. First, pre-trained CNN models (ResNet18, ResNet50, and ResNet101) were trained for bread wheat cultivars. Second, the features extracted from the fc1000 layer of the pre-trained CNN models ResNet18, ResNet50, and ResNet101 were classified using a support vector machine (SVM) classifier with different kernel features from machine learning techniques for classification with different variants. Finally, SVM methods were used in the second stage to classify the features obtained from the fc1000 layer of the pre-trained CNN models with an optimal set of features that can represent all features using the minimum redundancy maximum relevance (mRMR) feature selection algorithm.The accuracies obtained in the first, second, and last phases are as follows. In the first phase, the most successful method in classifying wheat grains was the ResNet18 model with 97.57%. In the second phase, the ResNet18 + ResNet50 + ResNet101 + Quadratic SVM model was the most successful model in classification using the features obtained from the ResNet CNN models with 94.08%.The accuracy for classification with the 1000 most effective features selected by the feature selection algorithm was 94.51%. Although the classification with features is slightly lower than deep learning, the classification time is much shorter and is 93%. This result confirms the great effectiveness of CNN models for wheat grain classification.

From programming-to-modeling-to-prompts smart ubiquitous applications

Since its introduction by Mark Weiser, ubiquitous computing has received increased interest in the dawn of technological advancement. Supported by wireless technology advancement, embedded systems, miniaturization, and the integration of various intelligent and communicative devise, context-aware ubiquitous applications actively and intelligently use rich contextual information to assist their users. However, their designs are subject to continuous changes imposed by external factors. Nowadays, software engineering, particularly in the fields of Model-Driven Engineering, displays a strong tendency towards developing applications for pervasive computing. This trend is also fueled by the rise of generative artificial intelligence, paving the way for a new generation of no-code development tools and models specifically trained on open-source code repositories to generate applications from their descriptions. The specificities of our approach lies in starting with a graphical model expressed using a domain-specific language (DSL) composed of symbols and formal notations. This allows for graphically instantiating and editing applications, guiding and assisting experts from various engineering fields in defining ubiquitous applications that are eventually transformed into peculiar models. We believe that creating intelligent models is the best way to promote software development efficiency. We have used and evaluated recurrent neural networks, leveraging the recurrence of processing the same contextual information collected within this model, and enabling iterative adaptation to future evolutions in ubiquitous systems. We propose a prototype instantiated by our meta-model which tracks the movements of individuals who were positive for COVID-19 and confirmed to be contagious. Different deep learning models and classical machine learning techniques are considered and compared for the task of detection/classification of COVID-19. Results obtained from all techniques were evaluated with confusion matrices, accuracy, precision, recall and F1-score. In summary, most of the results are very impressive. Our deep learning approach used a RNN architecture produced up to 92.1% accuracy. With the recent development of OpenAI Codex, optimized for programming languages, we provided the same requirements to the Codex model and asked it to generate the source code for the COVID-19 application, comparing it with the application generated by our workshop.

NuInsSeg: A fully annotated dataset for nuclei instance segmentation in H&E-stained histological images

In computational pathology, automatic nuclei instance segmentation plays an essential role in whole slide image analysis. While many computerized approaches have been proposed for this task, supervised deep learning (DL) methods have shown superior segmentation performances compared to classical machine learning and image processing techniques. However, these models need fully annotated datasets for training which is challenging to acquire, especially in the medical domain. In this work, we release one of the biggest fully manually annotated datasets of nuclei in Hematoxylin and Eosin (H&E)-stained histological images, called NuInsSeg. This dataset contains 665 image patches with more than 30,000 manually segmented nuclei from 31 human and mouse organs. Moreover, for the first time, we provide additional ambiguous area masks for the entire dataset. These vague areas represent the parts of the images where precise and deterministic manual annotations are impossible, even for human experts. The dataset and detailed step-by-step instructions to generate related segmentation masks are publicly available on the respective repositories.

Improving multilayer perceptron neural network using two enhanced moth-flame optimizers to forecast iron ore prices

Abstract The quality of the output produced by the multi-layer perceptron neural network depends on the careful selection of its weights and biases. The gradient descent technique is commonly used for choosing MLP’s optimal configuration, but it can suffer from being stuck in local optima and slow convergence toward promising regions in the search space. In this article, we propose two new optimization algorithms based on the moth-flame optimization algorithm (MFO), which mimics moths’ special navigation methods at night. We use these algorithms to enhance the performance of the training process of the MLP neural network. To demonstrate the effectiveness of our approach, we apply it to the problem of predicting iron ore prices, which plays an important role in the continuous development of the steel industry. We use a large number of features to predict the iron ore price, and we select a promising set of features using two feature reduction methods: Pearson’s correlation and a newly proposed categorized correlation. Surprisingly, new features not mentioned in the literature are discovered, and some are discarded. The time series dataset used has been extracted from several sources and pre-processed to fit the proposed model. We compare our two proposed MFO algorithms, the roulette wheel moth-flame optimization algorithm and the global best moth-flame optimization algorithm, against four swarm intelligence algorithms and five classical machine learning techniques when predicting the iron ore price. The results acquired indicate the superior performance of the suggested algorithms concerning prediction accuracy, root-mean-square error, mean-square error, average absolute relative deviation, and mean absolute error. Overall, our work presents a promising approach for improving the performance of MLP neural networks, and it demonstrates its effectiveness in the challenging problem of predicting iron ore prices.

Machine learning classifiers for fall detection leveraging LoRa communication network

Today, health monitoring relies heavily on technological advancements. This study proposes a low-power wide-area network (LPWAN) based, multinodal health monitoring system to monitor vital physiological data. The suggested system consists of two nodes, an indoor node, and an outdoor node, and the nodes communicate via long range (LoRa) transceivers. Outdoor nodes use an MPU6050 module, heart rate, oxygen pulse, temperature, and skin resistance sensors and transmit sensed values to the indoor node. We transferred the data received by the master node to the cloud using the Adafruit cloud service. The system can operate with a coverage of 4.5 km, where the optimal distance between outdoor sensor nodes and the indoor master node is 4 km. To further predict fall detection, various machine learning classification techniques have been applied. Upon comparing various classifier techniques, the decision tree method achieved an accuracy of 0.99864 with a training and testing ratio of 70:30. By developing accurate prediction models, we can identify high-risk individuals and implement preventative measures to reduce the likelihood of a fall occurring. Remote monitoring of the health and physical status of elderly people has proven to be the most beneficial application of this technology.

Selective Partitioned Regression for Accurate Kidney Health Monitoring

The number of people diagnosed with advanced stages of kidney disease have been rising every year. Early detection and constant monitoring are the only minimally invasive means to prevent severe kidney damage or kidney failure. We propose a cost-effective machine learning-based testing system that can facilitate inexpensive yet accurate kidney health checks. Our proposed framework, which was developed into an iPhone application, uses a camera-based bio-sensor and state-of-the-art classical machine learning and deep learning techniques for predicting the concentration of creatinine in the sample, based on colorimetric change in the test strip. The predicted creatinine concentration is then used to classify the severity of the kidney disease as healthy, intermediate, or critical. In this article, we focus on the effectiveness of machine learning models to translate the colorimetric reaction to kidney health prediction. In this setting, we thoroughly evaluated the effectiveness of our novel proposed models against state-of-the-art classical machine learning and deep learning approaches. Additionally, we executed a number of ablation studies to measure the performance of our model when trained using different meta-parameter choices. Our evaluation results indicate that our selective partitioned regression (SPR) model, using histogram of colors-based features and a histogram gradient boosted trees underlying estimator, exhibits much better overall prediction performance compared to state-of-the-art methods. Our initial study indicates that SPR can be an effective tool for detecting the severity of kidney disease using inexpensive lateral flow assay test strips and a smart phone-based application. Additional work is needed to verify the performance of the model in various settings.

Carpal Tunnel Syndrome Automated Diagnosis: A Motor vs. Sensory Nerve Conduction-Based Approach.

The objective of this study was to evaluate the effectiveness of machine learning classification techniques applied to nerve conduction studies (NCS) of motor and sensory signals for the automatic diagnosis of carpal tunnel syndrome (CTS). Two methodologies were tested. In the first methodology, motor signals recorded from the patients' median nerve were transformed into time-frequency spectrograms using the short-time Fourier transform (STFT). These spectrograms were then used as input to a deep two-dimensional convolutional neural network (CONV2D) for classification into two categories: patients and controls. In the second methodology, sensory signals from the patients' median and ulnar nerves were subjected to multilevel wavelet decomposition (MWD), and statistical and non-statistical features were extracted from the decomposed signals. These features were utilized to train and test classifiers. The classification target was set to three categories: normal subjects (controls), patients with mild CTS, and patients with moderate to severe CTS based on conventional electrodiagnosis results. The results of the classification analysis demonstrated that both methodologies surpassed previous attempts at automatic CTS diagnosis. The classification models utilizing the motor signals transformed into time-frequency spectrograms exhibited excellent performance, with average accuracy of 94%. Similarly, the classifiers based on the sensory signals and the extracted features from multilevel wavelet decomposition showed significant accuracy in distinguishing between controls, patients with mild CTS, and patients with moderate to severe CTS, with accuracy of 97.1%. The findings highlight the efficacy of incorporating machine learning algorithms into the diagnostic processes of NCS, providing a valuable tool for clinicians in the diagnosis and management of neuropathies such as CTS.

Applying machine learning classification techniques for disease diagnosis from medical imaging data using Transformer based Attention Guided CNN (TAGCNN)

Applying machine learning classification techniques for disease diagnosis from medical imaging data using Transformer based Attention Guided CNN (TAGCNN)

The five-factor model analysed by machine learning classification techniques

ABSTRACT This study compares the classification performance of traditional factor models and ML to explore the assumption of linearity in risk premiums. We apply ML techniques to the variables used in the Fama – French five-factor (FF5) model to classify the top 30% and bottom 30% of stocks based on whether their prices are expected to increase (winners) or decrease (losers), respectively, over a specified time period. The analysis showed that the ML model’s classification Accuracy outperforms the FF5 classification, which suggests that the higher returns obtained using the FF5 model can be attributed to its effectiveness in classifying stocks with exceptionally high or low returns, rather than the comprehensive use of its variables for overall classification purposes.

Maximizing efficiency in solar ammonia–water absorption refrigeration cycles: Exergy analysis, concentration impact, and advanced optimization with GBRT machine learning and FHO optimizer

A detailed analysis of energy and exergy is conducted on a single-effect solar ammonia–water (NH3–H2O) absorption refrigeration cycle (ARC) using TRNSYS and EES software. Considering the physical and chemical exergies, the exergy destruction rate (ĖD) in each component of the system is calculated, highlighting its contribution to the overall ĖD. The study explores the effects of varying refrigerant mass flow rate (ṁᵣ) and ammonia concentration in strong and weak solutions (Xs and Xw) on key performance parameters, including coefficient of performance (COP), exergy efficiency (ĖD), and overall ĖD across a range of generator temperatures (Tg). In this study, a gradient boosting regression tree (GBRT) is employed as a supervised machine-learning technique for classification and regression problems, utilizing boosting to enhance conventional decision tree predictions. The Fire Hawk Optimizer (FHO) approach is also utilized to optimize performance parameters, maximizing COP and ηηEwhile minimizing Tg and ĖD. The GBRT models are developed using available experimental and simulation data, revealing relationships between variables (ṁᵣ, Xs, Xw, and Tg) and outcomes (COP, ĖD, and overall ĖD). The results revealed that the generator exhibits considerable ĖD regardless of operating conditions, underscoring its pivotal role in the ARC. It emerges as the primary ĖD contributor (50 %), followed by the evaporator (17 %) and the absorber (15 %). However, ĖD associated with the recooler, pump, and expansion valves is negligible in comparison. Optimization results reveal that, when minimizing Tg and ĖD, the highest COP and ĖD at Tg of 373.15 K reach 0.8081 and 0.46, respectively.